Video codec standards and implementations are difficult to design for real-world performance as there is a limit to the amount of video data that can be used as part of the design process. Usually video codec standards are developed with a slow reference encoder and small number of video sequences covering a short segment of video content, e.g., 10s of seconds of video.
Real-time encoders are typically designed using larger sets of data, but still the design phase has access to a much smaller amount of data than an encoder in the field will encounter in its lifetime. This leads to suboptimal over-fitted designs, both of the original video codec standard and of implementations to the original video codec standard.